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ABSTRACT 



This report discusses the outcomes of a study that reviewed 
activities to ensure quality of care for adolescents receiving mental health 
services in the School Mental Health Program (SMHP) , based in the Department 
of Psychiatry at the University of Maryland School of Medicine. For this 
program a team of clinicians, as well as trainees in each area, provide 
services in Baltimore City piiblic schools. Services provided include 
individual, group, and family therapies; case management; collaboration, such 
as developing wraparound service plans; and primary, secondary and tertiary 
prevention activities, which include conducting mental health educational 
activities in school. Quality improvement activities for the SMHP were 
conducted in three phases: a structural appraisal, an evaluation of the 
treatment process, and an examination of treatment outcomes. Results 
indicate: (1) stakeholders were satisfied with services, however, all groups 

recognized a supply and demand problem; (2) students were commonly coping 
with bereavement and loss issues and needed to learn skills to manage anger, 
improve self-esteem, or to improve their coping skills in the classroom or in 
peer relationships; and (3) students were highly satisfied with the mental 
health services they received and valued the therapeutic relationship and 
skills they learned during therapy. (Contains 16 references.) (CR) 
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Introduction 

In the last decade several changes have redefined service provision for 
children and families. One paradigm shift has involved changes in service 
delivery. It has become standard practice to deliver comprehensive mental 
health services in community-based programs, such as schools (Eber & 
Nelson, 1997; Rosenblatt, & Attkisson, 1992, 1993). Another issue centers 
on the increasing need for mental health services for youth in an era of 
cutbacks and increasing demands for accountability (Hoagwood, Jensen, 
Petti, & Burns, 1996). Thus, it has become increasingly important to record 
indices of accountability demonstrating quality care for youth receiving 
mental health services. 

In this presentation, activities to ensure quality of care for adolescents 
receiving mental health services in the School Mental Health Program 
(SMHP) were reviewed. The SMHP is based in the Department of 
Psychiatry at the University of Maryland School of Medicine. For this 
program a team of clinicians, including licensed psychologists and social 
workers, as well as trainees in each area, provide services in Baltimore 
City public schools. A broad range of services are provided to youth, 
including: 1) individual, group, and family therapies; 2) case manage- 
ment; 3) collaboration, such as developing wraparound service plans, and; 
4) primary, secondary, and tertiary prevention activities, which include 
conducting mental health education activities in schools (Weist, 1997). 

Importance of Quality Improvement Activities 

Recognizing the importance of quality improvement (QI) activities, the 
Center for School Mental Health Assistance (CSMHA) , one of two national 
technical assistance centers to advance school mental health, convened a 
meeting of leaders in school mental health programs and quality assurance. 
Participants in this meeting identified key aspects of quality in school 
mental health programs. These key aspects are intended to: 

(1) Provide comprehensive direct clinical assessment and treatment 
services for underserved youth, (2) emphasize preventive programs that 
provide early identification and treatment for youth in need, (3) ensure 
that mental health programs have a strength or competency focus, versus 
an exclusive focus on reducing psychopathology, and (4) seek to 
maximize the impact of mental health services by improvement in 
collaborative efforts aimed at improving the global school environment 
(CSMHA, 1996, p. 4) 

Quality improvement activities for these programs should include 
development of a mission statement and program standards, conducting 
activities to assess program structure (e.g., developing staff training 
activities and orientation programs), developing a resource library, 
documenting structural variables (e.g., clinician experience), documenting 



Laura Nabors, Ph.D. 

Research Consultant 

Phone: 1-888-708-0980 

E-mail: Inabors @ umpsy. umaryland. edu 

Mark Weist, Ph.D. 

Director 

Phone: 410/328-8384 
E -mall: m weist @ umpsy. umaryland. edu 

Olga Acosta, Ph.D. 

Project Coordinator 

Phone: 1-888-708-0980 

E -mall: oacosta @ umpsy. umaryland. edu 

Nancy Tashman, Ph.D. 

Research Consultant 
Phone: 410/328-9183 
E-mail: 

ntashman @ umpsy. umaryland. edu 

Center for School Mental 
Health Assistance 

Department of Psychiatry, University of 
Maryland School of Medicine 
880 West Lexington Street, Floor 10 
Baltimore, Maryland 21201-1570 
1-888-708-0980 Fax 410/708-0984 



U.S. DEPARTMENT OF EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 

2 CATION AL RESOURCES INFORMATION 
CENTER (ERIC) 

lis document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to improve 

reproduction quality. 

• Points of view or opinions stated in this docu- 
ment do not necessarily represent official NIE 
position or policy. 




BEST COPY AVAILABLE 



2 1 



12th Annual Research Conference (1999) Proceedings — 107 



Nabors, Weist, Acosta &Tashman 



therapy process (e.g., supervision of trainees, record 
techniques and activities used in therapy sessions), 
and examining treatment outcome (e.g., using 
satisfaction surveys, behavioral checklists, inter- 
views; Nabors, Weist, Tashman, & Myers, 1999). 

Methods and Results 

Ql Activities in a Schooi-Based 
Mentai Heaith Program 

Quality improvement activities for the SMHP 
were conducted in three phases, beginning with: 

1) a structural appraisal (e.g., conducting a needs 
assessment) , followed by; 2) an evaluation of the 
treatment process (e.g., implementation of a peer 
review team), and then moving to; 3) an examination 
of treatment outcomes (Donabedian, 1980). 

Structural Appraisal The needs assessment was 
conducted with stakeholder focus groups of students, 
parents, clinicians, teachers, health center staff, and 
school administrators. Stakeholders were satisfied 
with services. However, all groups recognized a 
“supply and demand problem” related to the 
shortage of clinicians and the overwhelming number 
of students who could benefit from participating in 
therapy. Areas for improving the program centered 
on increasing education efforts for children and 
families so they had a better understanding of mented 
hecilth problems, increasing funding to hire more 
clinicians, and increasing family involvement in 
treatment (Nabors, Tashman, & Jackson, 1998; 

Nabors, Tashman, Myers, & Weist, 1998). 

Treatment Process Evaluation. The SMHP 
process of care was assessed through recording the 
activities of a Peer Review Team (PRT; Nabors, Lee, 
Tashman, Acosta, & Weist, 1999). The goals of the 
PRT were to evaluate therapy quality and to 
improve therapeutic process by suggesting 
interventions to enhance protective factors operating 
in the student’s life, or by providing ideas for 
wraparound services. Qualitative analyses of 
transcripts of PRT meetings with clinicians were 
used to determine relevant themes. Results indicated 
that students were commonly coping with bereave- 
ment and loss issues, needed to learn skills to 
manage anger, improve self-esteem, or to improve 
their coping skills in the classroom or in relation- 
ships with peers or family. Reviewers addressed 
these issues and provided recommendations about 
techniques to improve students’ abilities to cope 
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with traumatic experiences. During follow-up 
interviews, clinicians expressed appreciation for 
recommendations made by reviewers, especially 
those related to enhancing protective factors and 
improving case management services. 

Outcomes Evaluation. Program outcomes are 
assessed in several ways. For instance, program 
funders require information about the number of 
students seen in individual, group, and family 
sessions for each clinician over the course of the 
school year. Information on grades, attendance, and 
disciplinary encounters are recorded for students 
who have participated in what is considered an 
adequate "therapy dose,” i.e., attending four or 
more .sessions. 

The eight-item version of the Client Satisfaction 
Questionnaire (CSQ: Larsen, Attkisson, Hargreaves, 
& Nguyen, 1979) was used to examine student 
satisfaction with their "mental health counseling.” 
Results indicated that students were highly satisfied 
with the mental health services they received. 
Students valued the therapeutic relationship, 
"catharsis” associated with therapy, and skills they 
learned during therapy. Several factors influenced 
satisfaction ratings, including clinician training as 
well as student grades. Specifically, results of a 
regression analysis indicated that students were 
more satisfied with services provided by more 
experienced clinicians. Older students (juniors and 
seniors) were more satisfied with their therapy than 
underclassman. There was a trend for students with 
lower grades to feel less satisfied with mental health 
services than those with higher grades (Nabors, 
Reynolds, & Weist, 1999). Assessing adolescent 
satisfaction was useful in our program evaluation 
efforts. More research in this area is necessary. In 
future studies, researchers should continue 
examining children’s satisfaction with their mental 
health services as a routinely executed quality 
improvement activity. 

Future Program Evaiuations 

Funding provided by the Agency for Health Care 
Policy and Research (AHCPR) has enabled the 
SMHP to begin assessing other types of outcomes 
that may be more relevant for measuring change 
related to therapy participation. This stage of the 
program evaluation will be developed over the next 
two years. Key measures in evaluation efforts 
include the development of a tracking form (filled 
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out by clinicians), to assess “therapy dose” (e.g., 
number of sessions; session type — individual, 
group, family, crisis — and session length), and 
measurement of student behavioral and emotional 
functioning from student, parent, teacher, and 
clinician perspectives. 

Parent and student reports of student functioning 
will be assessed, using the Child Behavior Checklist 
(CBCL: Achenbach, 1991a) and Youth Self-Report 
Scale (YSR: Achenbach, 1991b). Clinician views of 
student functioning will be examined using the 
Child and Adolescent Functional Assessment Scale 
(CAFAS: Hodges, 1995) and the tracking form. 
Teachers will, complete a brief measure of student 
behavioral progress. Students also will fill out 
measures assessing perceptions of their clinician’s 
relationship skills and their satisfaction with 
services. Data on functional indicators such as 
grades and attendance will be collected and 
compared to ratings of student behavioral and 
emotional functioning. 

Conclusion 

Significance of Qi Programs 

Quality improvement activities for the SMHP 
have provided important accountability data. 

Results are disseminated to clinicians through 
newsletters. This contributes to clinical training 
efforts and helps to shape program evaluation 
activities. When QI activities are tied to training and 
evaluation efforts (e.g., to gather accountability data) 
there is a greater chance they will be incorporated 
into practice and become part of the way an 
organization functions. The QI activities reviewed 
can be incorporated into day-to-day operations for 
school mental health programs. Results from these 
activities can provide information for clinicians to 
improve quality of treatment and enhance the cost- 
effectiveness of treatment. Quality treatment can 
reduce referrals for psychiatric hospitalization or to 
the juvenile justice system, document the effective- 
ness of short-term therapy, and may increase the 
number of children graduating from school and 
entering the work force. 



References 

Achenbach, T. M. (1991a). Manual for the Child 
Behavior Checklist/4-18 and 1991 Profile. Burlington, VT: 
University of Vermont Department of Psychiatry. 

Achenbach, T. M. (1991b). Manual for the Youth 
Self-Report and 1991 Profile. Burlington, VT: 
University of Vermont Department of Psychiatry. 

Center for School Mental Health Assistance 
(1996). Quality assurance in expanded school mental 
health. Baltimore: Author. 

Donabedian, A. (1980). The definitions of quality 
and approaches to its assessment. Ann Arbor, MI: 

Health Administration Press. 

Eber, L., & Nelson, M. (1997). School-based 
wraparouiid planning: Integrating services for 
students with emotional and behavioreil needs. 
American Journal of Orthopsychiatry, 67, 385-395. 

Hoagwood, K., Jensen, P. S., Petti, T, & Burns, B. 

J. (1996). Outcomes of mental health care for chil- 
dren and adolescents: I. A comprehensive concep- 
tual model. Journal of the American Academy of Child 
and Adolescent Psychiatry, 35, 1055-1063. 

Hodges, K. (1995). Child and Adolescent Functional 
Assessment Scale (CAFAS; Version 2.15.96}. University 
Publishing: Ann Arbor, MI. 

Larsen, D. L., Attkisson, C. C., Hargreaves, W. A., 
& Nguyen, T D. (1979). Assessment of client/patient 
satisfaction: Development of a general scale. Evalua- 
tion and Program Planning, 2, 197-207. 

Nabors, L. A., Lee, T. M., Tashman, N. A., Acosta, 
O. M., Weist, M. D. (1999) Implementation of a peer 
review program in an expanded school mental health 
program. Manuscript submitted for publication. 

Nabors, L. A., Tashman, N., Myers, P, & Weist, 

M. (1 998) . Key processes for school mental health 
programs to advance. Poster presentation at the 11th 
Annual Conference: A System of Care for Children’s 
Mental Health: Expanding the Research Base (March 
8 to March 11, 1998). Tampa, FL. 



12th Annual Research Conference (1999) Proceedings — 109 



Nabors, Weist, Acosta &Tashman 



Nabors, L. A., Tashman, N., & Jackson, C. (1998). 
Quality assurance and evaluation in school-based mental 
health programs. Paper presentation at the 3rd 
National Conference on Advancing School-Based 
Mental Health Services, Virginia Beach, Virginia. 

Nabors, L., Reynolds, M. W., & Weist, M. D. 

Oune, 1999). Quality improvement activities in an 
expanded school mental health program. Fifth annual 
meeting of the National Assembly on School-Based 
Health Care. Washington, DC. 

Nabors, L., Weist, M., Tashman, N., & Myers, P. 
(1999). Quality assurance and school-based mental 
health services. Psychology in the Schools 36, 485-493. 

Rosenblatt, A., & Attkisson, C. C. (1992). Integrat- 
ing systems of care in California for youth with 
severe emotional disturbance I: A descriptive 
overview of the California AB377 evaluation project. 
Journal of Child and Family Studies, 1, 93-113. 

Rosenblatt, A., & Attkisson, C. C. (1993). Integrat- 
ing systems of care in California for youth with 
severe emotional disturbance III: Answers that lead 
to questions about out-of-home placements and the 
AB377 evaluation project. Journal of Child and Family 
Studies, 2, 119-141. 

Weist, M. D. (1997). Expanded school mental 
health services: A national movement in progress. In 
T. H. Ollendick, & R. J. Prinz (Eds.), Advances in 
clinical child psychology, Vol. 19 (pp. 319-352). New 
York: Plenum. 



IIO—Research and Training Center for Children’s Mental Health 




U.S. Department of Education 

Office of Educational Research and Improvement (OERIj 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 




NOTICE 

RF.PRODTTrTTON BASIS 




This document is covered by a signed “Reproduction Release 
(Blanket) form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 




This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release form 
(either “Specific Document” or “Blanket”). 



EFF-089 (9/97) 




